

<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<head>
  <meta charset="utf-8" />
  <meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />

  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  
  <title>RBD on Windows &mdash; Ceph Documentation</title>
  

  
  <link rel="stylesheet" href="../../_static/ceph.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/ceph.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/graphviz.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/css/custom.css" type="text/css" />

  
  

  
  

  

  
  <!--[if lt IE 9]>
    <script src="../../_static/js/html5shiv.min.js"></script>
  <![endif]-->
  
    
      <script type="text/javascript" id="documentation_options" data-url_root="../../" src="../../_static/documentation_options.js"></script>
        <script src="../../_static/jquery.js"></script>
        <script src="../../_static/_sphinx_javascript_frameworks_compat.js"></script>
        <script data-url_root="../../" id="documentation_options" src="../../_static/documentation_options.js"></script>
        <script src="../../_static/doctools.js"></script>
        <script src="../../_static/sphinx_highlight.js"></script>
    
    <script type="text/javascript" src="../../_static/js/theme.js"></script>

    
    <link rel="index" title="Index" href="../../genindex/" />
    <link rel="search" title="Search" href="../../search/" />
    <link rel="next" title="Ceph NVMe-oF Gateway" href="../nvmeof-overview/" />
    <link rel="prev" title="Ceph iSCSI 网关的监控" href="../iscsi-monitoring/" /> 
</head>

<body class="wy-body-for-nav">

   
  <header class="top-bar">
    <div role="navigation" aria-label="Page navigation">
  <ul class="wy-breadcrumbs">
      <li><a href="../../" class="icon icon-home" aria-label="Home"></a></li>
          <li class="breadcrumb-item"><a href="../">Ceph 块设备</a></li>
          <li class="breadcrumb-item"><a href="../rbd-integrations/">Ceph 块设备与第三方对接</a></li>
      <li class="breadcrumb-item active">RBD on Windows</li>
      <li class="wy-breadcrumbs-aside">
            <a href="../../_sources/rbd/rbd-windows.rst.txt" rel="nofollow"> View page source</a>
      </li>
  </ul>
  <hr/>
</div>
  </header>
  <div class="wy-grid-for-nav">
    
    <nav data-toggle="wy-nav-shift" class="wy-nav-side">
      <div class="wy-side-scroll">
        <div class="wy-side-nav-search"  style="background: #eee" >
          

          
            <a href="../../" class="icon icon-home"> Ceph
          

          
          </a>

          

          
<div role="search">
  <form id="rtd-search-form" class="wy-form" action="../../search/" method="get">
    <input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
    <input type="hidden" name="check_keywords" value="yes" />
    <input type="hidden" name="area" value="default" />
  </form>
</div>

          
        </div>

        
        <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
          
            
            
              
            
            
              <ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../../start/">Ceph 简介</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../install/">安装 Ceph</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../cephadm/">Cephadm</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../rados/">Ceph 存储集群</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../cephfs/">Ceph 文件系统</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="../">Ceph 块设备</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="../rados-rbd-cmds/">基本命令</a></li>
<li class="toctree-l2"><a class="reference internal" href="../rbd-operations/">运维</a></li>
<li class="toctree-l2 current"><a class="reference internal" href="../rbd-integrations/">对接</a><ul class="current">
<li class="toctree-l3"><a class="reference internal" href="../rbd-ko/">内核模块</a></li>
<li class="toctree-l3"><a class="reference internal" href="../qemu-rbd/">QEMU</a></li>
<li class="toctree-l3"><a class="reference internal" href="../libvirt/">libvirt</a></li>
<li class="toctree-l3"><a class="reference internal" href="../rbd-kubernetes/">Kubernetes</a></li>
<li class="toctree-l3"><a class="reference internal" href="../rbd-nomad/">Nomad</a></li>
<li class="toctree-l3"><a class="reference internal" href="../rbd-openstack/">OpenStack</a></li>
<li class="toctree-l3"><a class="reference internal" href="../rbd-cloudstack/">CloudStack</a></li>
<li class="toctree-l3"><a class="reference internal" href="../iscsi-overview/">LIO iSCSI Gateway</a></li>
<li class="toctree-l3 current"><a class="current reference internal" href="#">Windows</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#windows-service">Windows service</a></li>
<li class="toctree-l4"><a class="reference internal" href="#usage">Usage</a></li>
<li class="toctree-l4"><a class="reference internal" href="#troubleshooting">Troubleshooting</a></li>
</ul>
</li>
<li class="toctree-l3"><a class="reference internal" href="../nvmeof-overview/">NVMe-oF 网关</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../man/">手册页</a></li>
<li class="toctree-l2"><a class="reference internal" href="../api/">APIs</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../radosgw/">Ceph 对象网关</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../mgr/">Ceph 管理器守护进程</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../mgr/dashboard/">Ceph 仪表盘</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../monitoring/">监控概览</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../api/">API 文档</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../architecture/">体系结构</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../dev/developer_guide/">开发者指南</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../dev/internals/">Ceph 内幕</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../governance/">项目管理</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../foundation/">Ceph 基金会</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../ceph-volume/">ceph-volume</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../releases/general/">Ceph 版本（总目录）</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../releases/">Ceph 版本（索引）</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../security/">Security</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../hardware-monitoring/">硬件监控</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../glossary/">Ceph 术语</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../jaegertracing/">Tracing</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../translation_cn/">中文版翻译资源</a></li>
</ul>

            
          
        </div>
        
      </div>
    </nav>

    <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">

      
      <nav class="wy-nav-top" aria-label="top navigation">
        
          <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
          <a href="../../">Ceph</a>
        
      </nav>


      <div class="wy-nav-content">
        
        <div class="rst-content">
        
          <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
           <div itemprop="articleBody">
            
<div id="dev-warning" class="admonition note">
  <p class="first admonition-title">Notice</p>
  <p class="last">This document is for a development version of Ceph.</p>
</div>
  <div id="docubetter" align="right" style="padding: 5px; font-weight: bold;">
    <a href="https://pad.ceph.com/p/Report_Documentation_Bugs">Report a Documentation Bug</a>
  </div>

  
  <section id="rbd-on-windows">
<h1>RBD on Windows<a class="headerlink" href="#rbd-on-windows" title="Permalink to this heading"></a></h1>
<p>The <code class="docutils literal notranslate"><span class="pre">rbd</span></code> command can be used to create, remove, import, export, map or
unmap images exactly like it would on Linux. Make sure to check the
<a class="reference external" href="../rados-rbd-cmds">RBD basic commands</a> guide.</p>
<p><code class="docutils literal notranslate"><span class="pre">librbd.dll</span></code> is also available for applications that can natively use Ceph.</p>
<p>Please check the <a class="reference external" href="../../install/windows-install">installation guide</a> to get started.</p>
<section id="windows-service">
<h2>Windows service<a class="headerlink" href="#windows-service" title="Permalink to this heading"></a></h2>
<p>On Windows, <code class="docutils literal notranslate"><span class="pre">rbd-wnbd</span></code> daemons are managed by a centralized service. This allows
decoupling the daemons from the Windows session from which they originate. At
the same time, the service is responsible of recreating persistent mappings,
usually when the host boots.</p>
<p>Note that only one such service may run per host.</p>
<p>By default, all image mappings are persistent. Non-persistent mappings can be
requested using the <code class="docutils literal notranslate"><span class="pre">-onon-persistent</span></code> <code class="docutils literal notranslate"><span class="pre">rbd</span></code> flag.</p>
<p>Persistent mappings are recreated when the service starts, unless explicitly
unmapped. The service disconnects the mappings when being stopped. This also
allows adjusting the Windows service start order so that RBD images can be
mapped before starting services that may depend on it, such as VMMS.</p>
<p>In order to be able to reconnect the images, <code class="docutils literal notranslate"><span class="pre">rbd-wnbd</span></code> stores mapping
information in the Windows registry at the following location:
<code class="docutils literal notranslate"><span class="pre">SYSTEM\CurrentControlSet\Services\rbd-wnbd</span></code>.</p>
<p>The following command can be used to configure the service. Please update
the <code class="docutils literal notranslate"><span class="pre">rbd-wnbd.exe</span></code> path accordingly:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>New-Service -Name &quot;ceph-rbd&quot; `
            -Description &quot;Ceph RBD Mapping Service&quot; `
            -BinaryPathName &quot;c:\ceph\rbd-wnbd.exe service&quot; `
            -StartupType Automatic
</pre></div>
</div>
<p>Note that the Ceph MSI installer takes care of creating the <code class="docutils literal notranslate"><span class="pre">ceph-rbd</span></code>
Windows service.</p>
</section>
<section id="usage">
<h2>Usage<a class="headerlink" href="#usage" title="Permalink to this heading"></a></h2>
<section id="integration">
<h3>Integration<a class="headerlink" href="#integration" title="Permalink to this heading"></a></h3>
<p>RBD images can be exposed to the OS and host Windows partitions or they can be
attached to Hyper-V VMs in the same way as iSCSI disks.</p>
<p>Starting with Openstack Wallaby, the Nova Hyper-V driver can attach RBD Cinder
volumes to Hyper-V VMs.</p>
</section>
<section id="mapping-images">
<h3>Mapping images<a class="headerlink" href="#mapping-images" title="Permalink to this heading"></a></h3>
<p>The workflow and CLI is similar to the Linux counterpart, with a few
notable differences:</p>
<ul class="simple">
<li><p>device paths cannot be requested. The disk number and path will be picked by
Windows. If a device path is provided by the used when mapping an image, it
will be used as an identifier, which can also be used when unmapping the
image.</p></li>
<li><p>the <code class="docutils literal notranslate"><span class="pre">show</span></code> command was added, which describes a specific mapping.
This can be used for retrieving the disk path.</p></li>
<li><p>the <code class="docutils literal notranslate"><span class="pre">service</span></code> command was added, allowing <code class="docutils literal notranslate"><span class="pre">rbd-wnbd</span></code> to run as a Windows service.
All mappings are by default persistent, being recreated when the service
stops, unless explicitly unmapped. The service disconnects the mappings
when being stopped.</p></li>
<li><p>the <code class="docutils literal notranslate"><span class="pre">list</span></code> command also includes a <code class="docutils literal notranslate"><span class="pre">status</span></code> column.</p></li>
</ul>
<p>The purpose of the <code class="docutils literal notranslate"><span class="pre">service</span></code> mode is to ensure that mappings survive reboots
and that the Windows service start order can be adjusted so that RBD images can
be mapped before starting services that may depend on it, such as VMMS.</p>
<p>The mapped images can either be consumed by the host directly or exposed to
Hyper-V VMs.</p>
</section>
<section id="hyper-v-vm-disks">
<h3>Hyper-V VM disks<a class="headerlink" href="#hyper-v-vm-disks" title="Permalink to this heading"></a></h3>
<p>The following sample imports an RBD image and boots a Hyper-V VM using it:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span># Feel free to use any other image. This one is convenient to use for
# testing purposes because it&#39;s very small (~15MB) and the login prompt
# prints the pre-configured password.
wget http://download.cirros-cloud.net/0.5.1/cirros-0.5.1-x86_64-disk.img `
     -OutFile cirros-0.5.1-x86_64-disk.img

# We&#39;ll need to make sure that the imported images are raw (so no qcow2 or vhdx).
# You may get qemu-img from https://cloudbase.it/qemu-img-windows/
# You can add the extracted location to $env:Path or update the path accordingly.
qemu-img convert -O raw cirros-0.5.1-x86_64-disk.img cirros-0.5.1-x86_64-disk.raw

rbd import cirros-0.5.1-x86_64-disk.raw
# Let&#39;s give it a hefty 100MB size.
rbd resize cirros-0.5.1-x86_64-disk.raw --size=100MB

rbd device map cirros-0.5.1-x86_64-disk.raw

# Let&#39;s have a look at the mappings.
rbd device list
Get-Disk

$mappingJson = rbd-wnbd show cirros-0.5.1-x86_64-disk.raw --format=json
$mappingJson = $mappingJson | ConvertFrom-Json

$diskNumber = $mappingJson.disk_number

New-VM -VMName BootFromRBD -MemoryStartupBytes 512MB
# The disk must be turned offline before it can be passed to Hyper-V VMs
Set-Disk -Number $diskNumber -IsOffline $true
Add-VMHardDiskDrive -VMName BootFromRBD -DiskNumber $diskNumber
Start-VM -VMName BootFromRBD
</pre></div>
</div>
</section>
<section id="windows-partitions">
<h3>Windows partitions<a class="headerlink" href="#windows-partitions" title="Permalink to this heading"></a></h3>
<p>The following sample creates an empty RBD image, attaches it to the host and
initializes a partition:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>rbd create blank_image --size=1G
rbd device map blank_image -onon-persistent

$mappingJson = rbd-wnbd show blank_image --format=json
$mappingJson = $mappingJson | ConvertFrom-Json

$diskNumber = $mappingJson.disk_number

# The disk must be online before creating or accessing partitions.
Set-Disk -Number $diskNumber -IsOffline $false

# Initialize the disk, partition it and create a filesystem.
Get-Disk -Number $diskNumber | `
    Initialize-Disk -PassThru | `
    New-Partition -AssignDriveLetter -UseMaximumSize | `
    Format-Volume -Force -Confirm:$false

# Show the partition letter (for example, &quot;D:&quot; or &quot;F:&quot;):
(Get-Partition -DiskNumber $diskNumber).DriveLetter
</pre></div>
</div>
</section>
<section id="san-policy">
<h3>SAN policy<a class="headerlink" href="#san-policy" title="Permalink to this heading"></a></h3>
<p>The Windows SAN policy determines which disks will be automatically mounted.
The default policy (<code class="docutils literal notranslate"><span class="pre">offlineShared</span></code>) specifies that:</p>
<blockquote>
<div><p>All newly discovered disks that do not reside on a shared bus (such as SCSI
and iSCSI) are brought online and made read-write. Disks that are left
offline will be read-only by default.”</p>
</div></blockquote>
<p>Note that recent WNBD driver versions report rbd-wnbd disks as SAS, which is
also considered a shared bus. As a result, the disks will be offline and
read-only by default.</p>
<p>In order to turn a disk online (mounting the disk partitions) and clear the
read-only flag, use the following commands:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>Set-Disk -Number $diskNumber -IsOffline $false
Set-Disk -Number $diskNumber -IsReadOnly $false
</pre></div>
</div>
<p>Please check the <a class="reference internal" href="#limitations">Limitations</a> section to learn about the Windows limitations
that affect automatically mounted disks.</p>
<p>Windows documentation:</p>
<ul class="simple">
<li><p><a class="reference external" href="https://learn.microsoft.com/en-us/windows-hardware/customize/desktop/unattend/microsoft-windows-partitionmanager-sanpolicy">SAN policy reference</a></p></li>
<li><p><a class="reference external" href="https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/san">san command</a></p></li>
<li><p><a class="reference external" href="https://learn.microsoft.com/en-us/powershell/module/storage/set-storagesetting?view=windowsserver2022-ps">StorageSetting command</a></p></li>
</ul>
</section>
<section id="limitations">
<h3>Limitations<a class="headerlink" href="#limitations" title="Permalink to this heading"></a></h3>
<section id="csv-support">
<h4>CSV support<a class="headerlink" href="#csv-support" title="Permalink to this heading"></a></h4>
<p>At the moment, the Microsoft Failover Cluster can’t use WNBD disks as
Cluster Shared Volumes (CSVs) underlying storage. The main reason is that
<code class="docutils literal notranslate"><span class="pre">WNBD</span></code> and <code class="docutils literal notranslate"><span class="pre">rbd-wnbd</span></code> don’t support the <em>SCSI Persistent Reservations</em>
feature yet.</p>
</section>
<section id="hyper-v-disk-addressing">
<h4>Hyper-V disk addressing<a class="headerlink" href="#hyper-v-disk-addressing" title="Permalink to this heading"></a></h4>
<div class="admonition warning">
<p class="admonition-title">Warning</p>
<p>Hyper-V identifies passthrough VM disks by number instead of SCSI ID, although
the disk number can change across host reboots. This means that the VMs can end
up using incorrect disks after rebooting the host, which is an important
security concern. This issue also affects iSCSI and Fibre Channel disks.</p>
</div>
<p>There are a few possible ways of avoiding this Hyper-V limitation:</p>
<ul class="simple">
<li><p>use an NTFS/ReFS partition to store VHDX image files instead of directly
attaching the RBD image. This may slightly impact the IO performance.</p></li>
<li><p>use the Hyper-V <code class="docutils literal notranslate"><span class="pre">AutomaticStartAction</span></code> setting to prevent the VMs from
booting with the incorrect disks and have a script that updates VM disks
attachments before powering them back on. The <code class="docutils literal notranslate"><span class="pre">ElementName</span></code> field of the
<a class="reference external" href="https://docs.microsoft.com/en-us/windows/win32/hyperv_v2/msvm-storageallocationsettingdata">Msvm_StorageAllocationSettingData</a> <a class="reference external" href="https://docs.microsoft.com/en-us/windows/win32/wmisdk/wmi-start-page">WMI</a> class may be used to label VM
disk attachments.</p></li>
<li><p>use the Openstack Hyper-V driver, which automatically refreshes the VM disk
attachments before powering them back on.</p></li>
</ul>
</section>
<section id="automatically-mounted-disks">
<h4>Automatically mounted disks<a class="headerlink" href="#automatically-mounted-disks" title="Permalink to this heading"></a></h4>
<p>Disks that are marked as “online” or “writable” will remain so after being
reconnected (e.g. due to host reboots, Ceph service restarts, etc).</p>
<p>Unfortunately, Windows restores the disk status based on the disk number,
ignoring the disk unique identifier. However, the disk numbers can change
after being reconnected. This issue also affects iSCSI and Fibre Channel disks.</p>
<p>Let’s assume that the <a class="reference internal" href="#san-policy">SAN policy</a> is set to <code class="docutils literal notranslate"><span class="pre">offlineShared</span></code>, three
RBD images are attached and disk 1 is turned online. After a reboot, disk 1
will become online but it may now correspond to a different RBD image. This can
be an issue if the disk that was mounted on the host was actually meant for a
VM.</p>
</section>
</section>
</section>
<section id="troubleshooting">
<h2>Troubleshooting<a class="headerlink" href="#troubleshooting" title="Permalink to this heading"></a></h2>
<p>Please consult the <a class="reference external" href="../../install/windows-troubleshooting">Windows troubleshooting</a> page.</p>
</section>
</section>



<div id="support-the-ceph-foundation" class="admonition note">
  <p class="first admonition-title">Brought to you by the Ceph Foundation</p>
  <p class="last">The Ceph Documentation is a community resource funded and hosted by the non-profit <a href="https://ceph.io/en/foundation/">Ceph Foundation</a>. If you would like to support this and our other efforts, please consider <a href="https://ceph.io/en/foundation/join/">joining now</a>.</p>
</div>


           </div>
           
          </div>
          <footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
        <a href="../iscsi-monitoring/" class="btn btn-neutral float-left" title="Ceph iSCSI 网关的监控" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
        <a href="../nvmeof-overview/" class="btn btn-neutral float-right" title="Ceph NVMe-oF Gateway" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
    </div>

  <hr/>

  <div role="contentinfo">
    <p>&#169; Copyright 2016, Ceph authors and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0).</p>
  </div>

   

</footer>
        </div>
      </div>

    </section>

  </div>
  

  <script type="text/javascript">
      jQuery(function () {
          SphinxRtdTheme.Navigation.enable(true);
      });
  </script>

  
  
    
   

</body>
</html>